A Dual-Modal Presentation of Sequential Relationships in Texts

Based on Braddeley’s working memory model and research on human attention, this study intends to develop and validate a visual-auditory information presentation that: (1) minimizes the interference in information processing between the visual and auditory channels; and (2) improves the effectiveness of mental integration of information from different modalities. The Baddeley model suggests that imagery spatial information and verbal information can be concurrently held in different subsystems in human’s working memory. Accordingly, this research proposes a method to split textual information containing sequential relationships into a “graphic + voice” representation. We hypothesize that this dual-modal presentation will result in superior user comprehension performance and higher satisfaction as compared to pure textual display. An experiment was carried out to test the hypothesis. The independent variable was the presentation mode: textual display vs. visual-auditory presentation. The dependent variables were user performance and satisfaction. Twenty-eight subjects participated in this experiment. The results indicate that the “graphic + voice” presentation significantly improved both users’ information processing performance and their satisfaction. Results of this study will benefit the interface design of generic computer systems by alleviating the overabundance of information output in the visual channel. These findings may also help address the usability problems associated with small-screen computers.